{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Credits to: https://www.kaggle.com/cheongwoongkang/distilbert-qa-starter-cross-validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Formulation\n",
    "I formulate this task as an extractive question answering problem, such as SQuAD.  \n",
    "Given a question and context, the model is trained to find the answer spans in the context.\n",
    "\n",
    "Therefore, I use sentiment as question, text as context, selected_text as answer.\n",
    "- Question: sentiment\n",
    "- Context: text\n",
    "- Answer: selected_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameters & Options "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Hyperparameters\n",
    "batch_size = 64 # batch size\n",
    "lr = 5e-5 # learning rate\n",
    "epochs = 2 # number of epochs\n",
    "max_seq_len = 128 # max sequence length\n",
    "doc_stride = 64 # document stride\n",
    "\n",
    "# Options\n",
    "cross_validation = True # whether to use cross-validation\n",
    "K = 5 # number of CV splits\n",
    "post_processing = True # whether to use post-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import json\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing\n",
    "### Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [],
   "source": [
    "pd_train = pd.read_csv('/kaggle/input/tweet-sentiment-extraction/train.csv')\n",
    "pd_test = pd.read_csv('/kaggle/input/tweet-sentiment-extraction/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "np_train = np.array(pd_train)\n",
    "np_test = np.array(pd_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### K-fold Split\n",
    "Split the data into K folds for cross validation. Use the fixed random seed for reproducibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Given a data size, return the train/valid indicies for K splits.\n",
    "def split_data(num_examples, K):\n",
    "    np.random.seed(0)\n",
    "    idx = np.arange(num_examples)\n",
    "    np.random.shuffle(idx)\n",
    "    \n",
    "    boundary = num_examples // K\n",
    "    splits = [{} for _ in range(K)]\n",
    "    for i in range(K):\n",
    "        splits[i]['valid_idx'] = idx[i*boundary:(i+1)*boundary]\n",
    "        splits[i]['train_idx'] = np.concatenate((idx[:i*boundary], idx[(i+1)*boundary:]))\n",
    "\n",
    "        valid = np_train[splits[i]['valid_idx']]\n",
    "        d = {'neutral':0, 'positive':0, 'negative':0}\n",
    "        for line in valid:\n",
    "            d[line[-1]] += 1\n",
    "        print(d)\n",
    "        \n",
    "    return splits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'neutral': 2272, 'positive': 1688, 'negative': 1537}\n",
      "{'neutral': 2243, 'positive': 1687, 'negative': 1567}\n",
      "{'neutral': 2192, 'positive': 1733, 'negative': 1572}\n",
      "{'neutral': 2210, 'positive': 1748, 'negative': 1539}\n",
      "{'neutral': 2200, 'positive': 1726, 'negative': 1571}\n"
     ]
    }
   ],
   "source": [
    "splits = split_data(len(np_train), K)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert Data to SQuAD-style\n",
    "In this part, I convert the data into SQuAD-style.  \n",
    "Since I think most of the errors in the dataset are irreducible, I do not use additional preprocessing methods to handle them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert data to SQuAD-style\n",
    "def convert_data(data, directory, filename):\n",
    "    def find_all(input_str, search_str):\n",
    "        l1 = []\n",
    "        length = len(input_str)\n",
    "        index = 0\n",
    "        while index < length:\n",
    "            i = input_str.find(search_str, index)\n",
    "            if i == -1:\n",
    "                return l1\n",
    "            l1.append(i)\n",
    "            index = i + 1\n",
    "        return l1\n",
    "    \n",
    "    output = {}\n",
    "    output['version'] = 'v1.0'\n",
    "    output['data'] = []\n",
    "    \n",
    "    for line in data:\n",
    "        paragraphs = []\n",
    "        context = line[1]\n",
    "        qas = []\n",
    "        question = line[-1]\n",
    "        qid = line[0]\n",
    "        answers = []\n",
    "        answer = line[2]\n",
    "        if type(context) != str:\n",
    "            print(context, type(context))\n",
    "            continue\n",
    "        answer_starts = find_all(context, answer)\n",
    "        for answer_start in answer_starts:\n",
    "            answers.append({'answer_start': answer_start, 'text': answer})\n",
    "        qas.append({'question': question, 'id': qid, 'is_impossible': False, 'answers': answers})\n",
    "\n",
    "        paragraphs.append({'context': context, 'qas': qas})\n",
    "        output['data'].append({'title': 'None', 'paragraphs': paragraphs})\n",
    "\n",
    "    if not os.path.exists(directory):\n",
    "        os.makedirs(directory)\n",
    "\n",
    "    with open(os.path.join(directory, filename), 'w') as outfile:\n",
    "        json.dump(output, outfile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nan <class 'float'>\n",
      "nan <class 'float'>\n",
      "nan <class 'float'>\n",
      "nan <class 'float'>\n"
     ]
    }
   ],
   "source": [
    "# convert k-fold train data\n",
    "for i, split in enumerate(splits):\n",
    "    data = np_train[split['train_idx']]\n",
    "    directory = 'split_' + str(i+1)\n",
    "    filename = 'train.json'\n",
    "    convert_data(data, directory, filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "nan <class 'float'>\n"
     ]
    }
   ],
   "source": [
    "# convert original train/test data\n",
    "data = np_train\n",
    "directory = 'original'\n",
    "filename = 'train.json'\n",
    "convert_data(data, directory, filename)\n",
    "\n",
    "data = np_test\n",
    "filename = 'test.json'\n",
    "convert_data(data, directory, filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finetuning\n",
    "Install the pytorch-transformers package (v2.5.1) of [huggingface](https://github.com/huggingface/transformers)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing /kaggle/input/pytorchtransformers/transformers-2.5.1\r\n",
      "Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (1.18.2)\r\n",
      "Requirement already satisfied: tokenizers==0.5.2 in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (0.5.2)\r\n",
      "Requirement already satisfied: boto3 in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (1.12.32)\r\n",
      "Requirement already satisfied: filelock in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (3.0.12)\r\n",
      "Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (2.22.0)\r\n",
      "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (4.42.0)\r\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (2020.2.20)\r\n",
      "Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (0.1.85)\r\n",
      "Requirement already satisfied: sacremoses in /opt/conda/lib/python3.6/site-packages (from transformers==2.5.1) (0.0.38)\r\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.6/site-packages (from boto3->transformers==2.5.1) (0.9.5)\r\n",
      "Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.6/site-packages (from boto3->transformers==2.5.1) (0.3.3)\r\n",
      "Requirement already satisfied: botocore<1.16.0,>=1.15.32 in /opt/conda/lib/python3.6/site-packages (from boto3->transformers==2.5.1) (1.15.32)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->transformers==2.5.1) (2019.11.28)\r\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->transformers==2.5.1) (3.0.4)\r\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->transformers==2.5.1) (2.8)\r\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->transformers==2.5.1) (1.24.3)\r\n",
      "Requirement already satisfied: click in /opt/conda/lib/python3.6/site-packages (from sacremoses->transformers==2.5.1) (7.1.1)\r\n",
      "Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from sacremoses->transformers==2.5.1) (1.14.0)\r\n",
      "Requirement already satisfied: joblib in /opt/conda/lib/python3.6/site-packages (from sacremoses->transformers==2.5.1) (0.14.1)\r\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /opt/conda/lib/python3.6/site-packages (from botocore<1.16.0,>=1.15.32->boto3->transformers==2.5.1) (2.8.1)\r\n",
      "Requirement already satisfied: docutils<0.16,>=0.10 in /opt/conda/lib/python3.6/site-packages (from botocore<1.16.0,>=1.15.32->boto3->transformers==2.5.1) (0.15.2)\r\n",
      "Building wheels for collected packages: transformers\r\n",
      "  Building wheel for transformers (setup.py) ... \u001b[?25l-\b \b\\\b \b|\b \bdone\r\n",
      "\u001b[?25h  Created wheel for transformers: filename=transformers-2.5.1-py3-none-any.whl size=498878 sha256=a20b51141194483b94d6abdf91524847da250393ff1736662816c06908a08024\r\n",
      "  Stored in directory: /root/.cache/pip/wheels/b0/58/32/c9de6e928489e1884c4a65f0aaf8a6ebe5484617eaadbea3b6\r\n",
      "Successfully built transformers\r\n",
      "Installing collected packages: transformers\r\n",
      "  Attempting uninstall: transformers\r\n",
      "    Found existing installation: transformers 2.7.0\r\n",
      "    Uninstalling transformers-2.7.0:\r\n",
      "      Successfully uninstalled transformers-2.7.0\r\n",
      "Successfully installed transformers-2.5.1\r\n"
     ]
    }
   ],
   "source": [
    "!cd /kaggle/input/pytorchtransformers/transformers-2.5.1; pip install ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cross-Validation\n",
    "Finetune QA models for cross-validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_kg_hide-output": false
   },
   "outputs": [],
   "source": [
    "def run_script(train_file, predict_file, batch_size=16, lr=5e-5, epochs=2, max_seq_len=128, doc_stride=64):\n",
    "    !python /kaggle/input/pytorchtransformers/transformers-2.5.1/examples/run_squad.py \\\n",
    "    --model_type distilbert \\\n",
    "    --model_name_or_path distilbert-base-uncased \\\n",
    "    --cache_dir /kaggle/input/cached-distilbert-base-uncased/cache \\\n",
    "    --do_lower_case \\\n",
    "    --do_train \\\n",
    "    --do_eval \\\n",
    "    --train_file=$train_file \\\n",
    "    --predict_file=$predict_file \\\n",
    "    --overwrite_cache \\\n",
    "    --learning_rate=$lr \\\n",
    "    --num_train_epochs=$epochs \\\n",
    "    --max_seq_length=$max_seq_len \\\n",
    "    --doc_stride=$doc_stride \\\n",
    "    --output_dir ./results \\\n",
    "    --overwrite_output_dir \\\n",
    "    --per_gpu_eval_batch_size=$batch_size \\\n",
    "    --per_gpu_train_batch_size=$batch_size \\\n",
    "    --save_steps=100000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_kg_hide-output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-04-05 08:29:31.117090: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 21989/21989 [00:01<00:00, 12905.34it/s]\r\n",
      "convert squad examples to features: 100%|█| 21989/21989 [00:44<00:00, 488.66it/s\r\n",
      "add example index and unique id: 100%|█| 21989/21989 [00:00<00:00, 525995.35it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:01<06:38,  1.16s/it]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:01<05:15,  1.09it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<04:17,  1.32it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:02<03:37,  1.56it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:02<03:08,  1.79it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:48,  2.00it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:03<02:34,  2.18it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:03<02:24,  2.33it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:04<02:17,  2.44it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:04<02:12,  2.53it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:09,  2.58it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:05<02:06,  2.62it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:05<02:04,  2.66it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:03,  2.68it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:06<02:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:06<02:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:07<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:07<01:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:08<01:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:08<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:09<01:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:09<01:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:10<01:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:11<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:11<01:54,  2.75it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:12<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:12<01:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:13<01:52,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:13<01:52,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:52,  2.75it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:14<01:51,  2.75it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:51,  2.75it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:15<01:50,  2.75it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:15<01:50,  2.75it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:16<01:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:16<01:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:18<01:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:19<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:19<01:46,  2.75it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.75it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:20<01:45,  2.75it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:20<01:45,  2.75it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:21<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:22<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:23<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:23<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:24<01:41,  2.75it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:24<01:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:25<01:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:26<01:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:26<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:27<01:41,  2.68it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:27<01:40,  2.69it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:29<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:38,  2.70it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:30<01:38,  2.68it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:30<01:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:31<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:31<01:36,  2.70it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:36,  2.68it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:32<01:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:33<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:34<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:36<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:37<01:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:37<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:38<01:31,  2.66it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:38<01:30,  2.67it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:29,  2.69it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:39<01:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:40<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:40<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:41<01:26,  2.69it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:41<01:26,  2.70it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:42<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:42<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:44<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:44<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:45<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:45<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:46<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:47<01:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:47<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:48<01:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:48<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:49<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:49<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:50<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:51<01:15,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:51<01:14,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:52<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:52<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:53<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:53<01:12,  2.75it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:54<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:55<01:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:55<01:11,  2.75it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:56<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:56<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:58<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:59<01:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:59<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:01<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:02<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:02<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:03<01:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:03<01:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:04<01:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:05<01:00,  2.76it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:00,  2.75it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:06<01:00,  2.75it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:06<01:00,  2.75it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:07<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<00:58,  2.75it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.75it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:08<00:58,  2.75it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:09<00:57,  2.75it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:10<00:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:10<00:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:11<00:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:12<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:13<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:15<00:51,  2.75it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:50,  2.75it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:50,  2.75it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:16<00:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:17<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.70it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:18<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:19<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:20<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:21<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:22<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:23<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:24<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:25<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:26<00:41,  2.69it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:27<00:40,  2.69it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:28<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:28<00:38,  2.70it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:29<00:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:38,  2.62it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:39,  2.55it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:30<00:38,  2.59it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:37,  2.64it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:31<00:37,  2.65it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:31<00:36,  2.66it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:32<00:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:32<00:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:34<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.68it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:36<00:31,  2.68it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:36<00:31,  2.68it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.69it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:37<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:39<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:39<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:40<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:41<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:42<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:42<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:43<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:43<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:44<00:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:45<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:46<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:46<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:47<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:47<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:49<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:49<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:50<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:50<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:51<00:15,  2.69it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:51<00:15,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:15,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:52<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:53<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:53<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:54<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:54<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:55<00:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:56<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:56<00:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:57<00:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:58<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:59<00:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [02:00<00:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [02:00<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:01<00:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:01<00:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.75it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:03<00:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:04<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:04<00:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:05<00:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:05<00:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:06<00:00,  2.75it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.75it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.71it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:06<02:06, 126.84s/it]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:01<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:02<02:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:00,  2.74it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:00,  2.74it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:05<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<01:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<01:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:06<01:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:58,  2.68it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:09<01:57,  2.70it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:10<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:53,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:52,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:12<01:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:13<01:51,  2.75it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:15<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:16<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:17<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:19<01:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:20<01:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:45,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:45,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:22<01:49,  2.58it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:48,  2.59it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:46,  2.64it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:23<01:45,  2.64it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:45,  2.64it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:43,  2.68it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:42,  2.69it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:26<01:39,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:27<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:33<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:37<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:38<01:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:41<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:42<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:44<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:45<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:48<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:49<01:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:13,  2.75it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:52<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:13,  2.69it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.74it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:56<01:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [00:59<01:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:00<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:03<01:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:03,  2.67it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:03,  2.66it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:02,  2.68it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<01:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:06<00:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:13<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:14<00:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:52,  2.67it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:51,  2.67it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.70it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:17<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:18<00:47,  2.71it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:21<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.70it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:43,  2.67it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:43,  2.69it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.69it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:42,  2.70it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:24<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:25<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:40,  2.68it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.69it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.69it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:38,  2.69it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:28<00:38,  2.68it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.68it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:37,  2.69it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.70it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:32<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:39<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:43<00:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.75it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:47<00:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.68it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.69it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:50<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:54<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:58<00:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.69it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.73it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:05<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.73it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [04:13<00:00, 126.53s/it]\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 13676.06it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:47<00:00, 582.25it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 579522.96it/s\r\n",
      "Evaluating: 100%|█████████████████████████████| 430/430 [00:54<00:00,  7.92it/s]\r\n",
      "2020-04-05 08:40:53.568549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 21988/21988 [00:01<00:00, 12639.40it/s]\r\n",
      "convert squad examples to features: 100%|█| 21988/21988 [00:45<00:00, 481.50it/s\r\n",
      "add example index and unique id: 100%|█| 21988/21988 [00:00<00:00, 577419.93it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<03:28,  1.65it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<03:02,  1.87it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:45,  2.07it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:32,  2.22it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:02<02:23,  2.36it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:17,  2.46it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:12,  2.54it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:03<02:09,  2.59it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:07,  2.64it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:05,  2.67it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:04,  2.68it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:02,  2.70it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:06<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<01:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:09<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:56,  2.67it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:58,  2.61it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:13<01:56,  2.64it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:56,  2.65it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:55,  2.65it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:54,  2.67it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:53,  2.69it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:50,  2.74it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:16<01:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:20<01:48,  2.68it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:23<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:24<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:40,  2.75it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:40,  2.75it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:39,  2.75it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:39,  2.75it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:38,  2.75it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:27<01:38,  2.75it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:38,  2.75it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:37,  2.75it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:35,  2.75it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:35,  2.75it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:36,  2.70it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:31<01:37,  2.67it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:37,  2.65it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:37,  2.64it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:36,  2.67it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:35<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:38<01:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:41<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:42<01:26,  2.67it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:25,  2.69it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:21,  2.75it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:45<01:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:19,  2.75it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:19,  2.76it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:18,  2.76it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:18,  2.76it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:18,  2.76it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:17,  2.76it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:17,  2.76it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:17,  2.76it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:16,  2.76it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:49<01:16,  2.76it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:15,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.76it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:13,  2.75it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:53<01:14,  2.67it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:14,  2.67it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:56<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:07,  2.75it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:07,  2.75it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:07,  2.76it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [01:00<01:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.75it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:57,  2.75it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.75it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:55,  2.67it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:54,  2.67it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.69it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:53,  2.68it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:13<00:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:52,  2.69it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:52,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:52,  2.67it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:46,  2.75it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:46,  2.76it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.76it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:45,  2.75it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:44,  2.75it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:24<00:41,  2.75it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.75it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:32<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:35<00:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.70it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:39<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:43<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.75it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.75it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:18,  2.75it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.75it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:47<00:18,  2.75it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:14,  2.75it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:54<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:58<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.70it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.70it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.70it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.74it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:05<00:00,  2.74it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.73it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:06<02:06, 126.17s/it]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<02:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:01<02:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:02<02:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:03,  2.70it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:02,  2.70it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:05<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:07,  2.56it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<02:07,  2.56it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<02:04,  2.60it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<02:02,  2.65it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<02:02,  2.64it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<02:01,  2.66it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:59,  2.69it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:08<01:58,  2.70it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:54,  2.75it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:56,  2.67it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:55,  2.69it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:12<01:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:15<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:49,  2.69it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:48,  2.71it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:19<01:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:22<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.75it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:23<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:41,  2.70it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:41,  2.69it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:26<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:30<01:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:33<01:33,  2.70it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.70it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:30,  2.68it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:37<01:29,  2.69it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:41<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.75it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:22,  2.75it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:44<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:19,  2.68it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:19,  2.69it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:48<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.69it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:52<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.73it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:56<01:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [00:59<01:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<01:05,  2.51it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<01:04,  2.53it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<01:02,  2.58it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<01:01,  2.61it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<01:00,  2.65it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:59,  2.68it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:58,  2.70it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.70it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.75it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.75it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:38,  2.75it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:35<00:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.75it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.70it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.75it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.71it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.71it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.72it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [04:12<00:00, 126.23s/it]\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 13310.59it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:47<00:00, 573.45it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 554222.25it/s\r\n",
      "Evaluating: 100%|█████████████████████████████| 430/430 [00:54<00:00,  7.93it/s]\r\n",
      "2020-04-05 08:52:12.657294: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 21988/21988 [00:01<00:00, 11139.58it/s]\r\n",
      "convert squad examples to features: 100%|█| 21988/21988 [00:46<00:00, 476.00it/s\r\n",
      "add example index and unique id: 100%|█| 21988/21988 [00:00<00:00, 585083.40it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<03:29,  1.64it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<03:03,  1.86it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:45,  2.06it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:33,  2.21it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:02<02:24,  2.35it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:17,  2.46it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:12,  2.54it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:03<02:10,  2.58it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:08,  2.61it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:05,  2.65it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:04,  2.67it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:03,  2.68it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:05<02:03,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<02:00,  2.70it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<01:59,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<02:00,  2.69it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:59,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:09<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:58,  2.68it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:59,  2.65it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:59,  2.65it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:57,  2.67it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:57,  2.68it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:12<01:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:54,  2.70it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:13<01:53,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:50,  2.75it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:49,  2.75it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:16<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:19<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:20<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:48,  2.66it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:47,  2.66it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:47,  2.67it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:22<01:46,  2.69it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:45,  2.70it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:23<01:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:24<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:26<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:27<01:40,  2.69it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.70it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:30<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:31<01:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:33<01:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:31,  2.70it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:31,  2.70it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:37<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:38<01:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:40<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:41<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:42<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:45<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:47<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:48<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:49<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:51<01:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:16,  2.67it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:17,  2.65it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:52<01:16,  2.67it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:15,  2.68it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.69it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:53<01:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:54<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:55<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:56<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:10,  2.70it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:10,  2.69it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:09,  2.70it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:58<01:09,  2.69it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:09,  2.69it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:08,  2.70it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:59<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:01<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:02<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:04,  2.70it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:03<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:05<01:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:06<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:08<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:09<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:10<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:12<00:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.70it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:54,  2.70it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:13<00:53,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:16<00:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:17<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:49,  2.70it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.69it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.70it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:19<00:48,  2.69it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.70it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:20<00:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:21<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:23<00:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:24<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:26<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:27<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:28<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:29<00:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:30<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:31<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:32<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:33<00:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:34<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:36<00:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:36<00:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:37<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:38<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:39<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:40<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:42<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:43<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:44<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:45<00:21,  2.71it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.70it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:21,  2.70it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:46<00:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:47<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:47<00:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.70it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:50<00:17,  2.68it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:50<00:16,  2.68it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.68it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:51<00:16,  2.55it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:51<00:16,  2.56it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:16,  2.56it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:52<00:15,  2.60it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.64it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:53<00:14,  2.66it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:53<00:13,  2.68it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.69it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:54<00:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:54<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:55<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:56<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:56<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:58<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:59<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.69it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [02:00<00:07,  2.68it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [02:00<00:06,  2.66it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:01<00:05,  2.67it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:01<00:05,  2.69it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.70it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:02<00:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:02<00:04,  2.63it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:03<00:04,  2.65it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:03<00:03,  2.67it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.68it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:04<00:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:04<00:02,  2.70it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:05<00:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:05<00:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:06<00:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:06<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.73it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.71it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:06<02:06, 126.96s/it]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<02:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<02:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:01<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:02<02:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:02,  2.70it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:05<02:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:06<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:08<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:09<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:12<01:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:13<01:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:54,  2.65it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:54,  2.65it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:15<01:53,  2.66it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:16<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:19<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:22<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:23<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:43,  2.66it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:42,  2.66it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:41,  2.68it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:26<01:40,  2.70it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.70it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:30<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:33<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:31,  2.70it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:31,  2.70it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:37<01:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:26,  2.71it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:41<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:26,  2.58it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:26,  2.58it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:45<01:24,  2.62it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:23,  2.65it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:22,  2.68it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:21,  2.69it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:19,  2.70it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:19,  2.69it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.70it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:18,  2.69it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:49<01:18,  2.70it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:16,  2.70it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:18,  2.64it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:17,  2.66it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:16,  2.68it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:15,  2.69it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:52<01:15,  2.69it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:14,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:53<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:11,  2.70it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:56<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.72it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:03<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:06<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<01:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<01:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<00:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:10<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:56,  2.70it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:13<00:53,  2.70it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:17<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:24<00:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.70it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:28<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:31<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:32<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:35<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:36<00:30,  2.70it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:38<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:39<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:26,  2.71it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:42<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:43<00:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:23,  2.61it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:22,  2.62it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:45<00:22,  2.65it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.67it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:21,  2.69it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:46<00:20,  2.70it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.68it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:20,  2.67it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:47<00:19,  2.68it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.67it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:48<00:18,  2.69it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:50<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.70it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:51<00:15,  2.70it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:15,  2.66it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:52<00:14,  2.69it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:53<00:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:13,  2.69it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:54<00:12,  2.65it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.67it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:55<00:11,  2.68it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.70it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:11,  2.69it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:56<00:10,  2.70it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:57<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:59<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [02:00<00:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.69it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.70it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:01<00:05,  2.70it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:02<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:02<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:03<00:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:04<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:05<00:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.69it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:06<00:00,  2.70it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.71it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.72it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [04:13<00:00, 126.82s/it]\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 13157.33it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:49<00:00, 556.21it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 530466.48it/s\r\n",
      "Evaluating: 100%|█████████████████████████████| 430/430 [00:54<00:00,  7.90it/s]\r\n",
      "2020-04-05 09:03:41.999797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 21988/21988 [00:01<00:00, 12072.52it/s]\r\n",
      "convert squad examples to features: 100%|█| 21988/21988 [00:46<00:00, 471.67it/s\r\n",
      "add example index and unique id: 100%|█| 21988/21988 [00:00<00:00, 557145.87it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<03:29,  1.63it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<03:03,  1.86it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:45,  2.06it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:32,  2.23it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:02<02:24,  2.35it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:17,  2.45it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:12,  2.54it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:03<02:11,  2.56it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:08,  2.61it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:06,  2.65it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:04,  2.67it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:03,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:05<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:09<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:13<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:53,  2.70it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:16<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:50,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.75it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:20<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:45,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:45,  2.69it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:45,  2.68it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:23<01:44,  2.70it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:24<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:43,  2.69it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:43,  2.68it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:43,  2.68it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:42,  2.69it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:41,  2.69it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:27<01:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:35,  2.75it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:31<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:34<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:38<01:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:42<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:45<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:49<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:53<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:56<01:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [01:00<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:03<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:58,  2.70it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:58,  2.69it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:58,  2.69it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:53,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:45,  2.69it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:45,  2.66it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:45,  2.64it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:44,  2.66it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:43,  2.66it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:24<00:43,  2.68it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.70it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:39,  2.66it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:28<00:39,  2.68it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.68it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:38,  2.67it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:37,  2.69it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:31<00:36,  2.69it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:32<00:34,  2.69it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:36<00:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.75it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:38<00:28,  2.75it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:39<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:42<00:24,  2.70it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.70it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:43<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.71it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:46<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:47<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:50<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:51<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:53<00:13,  2.67it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.68it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:13,  2.69it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:54<00:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:56<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:58<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [02:00<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:01<00:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:02<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:04<00:02,  2.68it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:05<00:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.72it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:06<02:06, 126.52s/it]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<02:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<02:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:01<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:02<02:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:00,  2.74it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:05<01:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<01:59,  2.75it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<01:58,  2.75it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:06<01:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:08<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:09<01:55,  2.75it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:10<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:53,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:53,  2.75it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:12<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:13<01:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:14<01:50,  2.74it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:15<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:16<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:17<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:47,  2.71it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:47,  2.70it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:19<01:47,  2.70it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:20<01:46,  2.69it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:46,  2.68it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:45,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:22<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:23<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:24<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:39,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:26<01:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:27<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:30<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:32<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:33<01:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:31,  2.75it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:34<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:35<01:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:37<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:27,  2.75it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.75it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:38<01:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:39<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:41<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:42<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:43<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:44<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:45<01:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:46<01:18,  2.75it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:17,  2.75it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:47<01:17,  2.75it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:48<01:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:49<01:15,  2.75it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:50<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:14,  2.70it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:52<01:14,  2.68it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:14,  2.68it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:13,  2.69it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:53<01:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:54<01:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.75it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.74it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:56<01:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:57<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [00:59<01:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:00<01:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:01<01:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:03<01:02,  2.70it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:04<01:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:05<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:07<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:08<00:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.75it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:09<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:11<00:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:12<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:13<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:13<00:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:14<00:52,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:15<00:51,  2.68it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:51,  2.66it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:50,  2.69it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:16<00:49,  2.69it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:49,  2.67it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:17<00:49,  2.67it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:49,  2.67it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:48,  2.69it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:18<00:47,  2.71it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:19<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:21<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:22<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:23<00:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:24<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:25<00:40,  2.70it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:26<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:30<00:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:33<00:32,  2.75it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:34<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:35<00:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:29,  2.74it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:36<00:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:29,  2.69it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.70it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:37<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:39<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:26,  2.71it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:40<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:41<00:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:43<00:22,  2.75it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:21,  2.75it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.75it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:45<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:47<00:18,  2.70it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:18,  2.70it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:48<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:50<00:15,  2.75it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:14,  2.75it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:51<00:14,  2.75it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:54<00:11,  2.75it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:10,  2.76it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.75it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:55<00:10,  2.75it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.76it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:56<00:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.75it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.75it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.70it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:58<00:07,  2.66it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:07,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.68it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [01:59<00:06,  2.70it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:02<00:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:03<00:02,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.75it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.74it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:05<00:00,  2.74it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:05<00:00,  2.73it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [04:12<00:00, 126.24s/it]\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 13494.01it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:47<00:00, 572.66it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 514139.89it/s\r\n",
      "Evaluating: 100%|█████████████████████████████| 430/430 [00:54<00:00,  7.94it/s]\r\n",
      "2020-04-05 09:15:01.944506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 21988/21988 [00:01<00:00, 12757.53it/s]\r\n",
      "convert squad examples to features: 100%|█| 21988/21988 [00:45<00:00, 488.37it/s\r\n",
      "add example index and unique id: 100%|█| 21988/21988 [00:00<00:00, 535254.53it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<03:32,  1.61it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<03:05,  1.84it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:48,  2.03it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:34,  2.20it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:02<02:24,  2.34it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:18,  2.44it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:13,  2.52it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:03<02:09,  2.59it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:07,  2.64it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:05,  2.65it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:04,  2.68it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:03,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:05<02:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:04,  2.66it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:04,  2.65it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:06<02:02,  2.68it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:01,  2.70it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<01:59,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:59,  2.70it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:08<02:00,  2.69it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<02:00,  2.68it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:59,  2.70it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:09<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:10<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:56,  2.69it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:12<01:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:54,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:13<01:53,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:14<01:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:16<01:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:49,  2.75it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:17<01:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:47,  2.75it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:19<01:48,  2.70it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:47,  2.71it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:20<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:21<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:23<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:24<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:25<01:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:39,  2.75it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:39,  2.75it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:27<01:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:38,  2.75it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:36,  2.75it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:37,  2.69it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:31<01:36,  2.69it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:32<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:34<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:35<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:29,  2.75it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:36<01:29,  2.75it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:28,  2.75it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:28,  2.75it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:28,  2.74it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.75it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:38<01:27,  2.75it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:39<01:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.75it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:42<01:23,  2.75it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.75it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:43<01:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:44<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:46<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:19,  2.75it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:18,  2.75it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:17,  2.75it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:17,  2.75it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:48<01:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:52<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:54<01:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:55<01:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:57<01:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:58<01:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [00:59<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:01<01:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:03,  2.70it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:04<01:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:05<01:05,  2.55it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:04,  2.57it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:03,  2.61it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<01:02,  2.64it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<01:01,  2.67it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:07<01:00,  2.68it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<01:00,  2.67it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<01:00,  2.65it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:08<00:59,  2.67it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:58,  2.69it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:10<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:11<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:12<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:52,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:50,  2.74it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:50,  2.74it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:19<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:46,  2.75it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:45,  2.75it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:45,  2.75it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:45,  2.74it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:44,  2.74it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:44,  2.75it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:22<00:43,  2.75it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:43,  2.75it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.75it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:23<00:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:25<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:26<00:40,  2.74it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:29<00:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:30<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.75it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:33<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:34<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:36<00:30,  2.70it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.69it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:30,  2.70it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:37<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:40<00:26,  2.75it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.75it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:41<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:44<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:48<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:51<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:52<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:55<00:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.74it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:59<00:07,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:06,  2.75it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.75it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:05,  2.75it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.75it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.74it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.74it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.68it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.65it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.67it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.72it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:06<02:06, 126.27s/it]\r\n",
      "Iteration:   0%|                                        | 0/344 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/344 [00:00<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 2/344 [00:00<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 3/344 [00:01<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/344 [00:01<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 5/344 [00:01<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 6/344 [00:02<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 7/344 [00:02<02:04,  2.70it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 8/344 [00:02<02:04,  2.69it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                               | 9/344 [00:03<02:04,  2.69it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 10/344 [00:03<02:04,  2.68it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 11/344 [00:04<02:03,  2.69it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 12/344 [00:04<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 13/344 [00:04<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 14/344 [00:05<02:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 15/344 [00:05<02:03,  2.67it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 16/344 [00:05<02:02,  2.69it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 17/344 [00:06<02:03,  2.64it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 18/344 [00:06<02:01,  2.67it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 19/344 [00:07<02:00,  2.69it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 20/344 [00:07<01:59,  2.70it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 21/344 [00:07<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 22/344 [00:08<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 23/344 [00:08<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 24/344 [00:08<01:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 25/344 [00:09<01:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▎                            | 26/344 [00:09<01:56,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 27/344 [00:09<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 28/344 [00:10<01:55,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 29/344 [00:10<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 30/344 [00:11<01:54,  2.74it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 31/344 [00:11<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 32/344 [00:11<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 33/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 34/344 [00:12<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 35/344 [00:12<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 36/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 37/344 [00:13<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 38/344 [00:13<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 39/344 [00:14<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 40/344 [00:14<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 41/344 [00:15<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 42/344 [00:15<01:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▉                           | 43/344 [00:15<01:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 44/344 [00:16<01:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 45/344 [00:16<01:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 46/344 [00:16<01:49,  2.73it/s]\u001b[A\r\n",
      "Iteration:  14%|████▏                          | 47/344 [00:17<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 48/344 [00:17<01:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 49/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 50/344 [00:18<01:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 51/344 [00:18<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 52/344 [00:19<01:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 53/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 54/344 [00:19<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 55/344 [00:20<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 56/344 [00:20<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 57/344 [00:20<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 58/344 [00:21<01:44,  2.73it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 59/344 [00:21<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 60/344 [00:22<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 61/344 [00:22<01:43,  2.75it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 62/344 [00:22<01:43,  2.74it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 63/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 64/344 [00:23<01:42,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 65/344 [00:23<01:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 66/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|██████                         | 67/344 [00:24<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 68/344 [00:24<01:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 69/344 [00:25<01:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 70/344 [00:25<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 71/344 [00:26<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 72/344 [00:26<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 73/344 [00:26<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 74/344 [00:27<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 75/344 [00:27<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 76/344 [00:27<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 77/344 [00:28<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 78/344 [00:28<01:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 79/344 [00:29<01:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 80/344 [00:29<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                       | 81/344 [00:29<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 82/344 [00:30<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▍                       | 83/344 [00:30<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▌                       | 84/344 [00:30<01:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▋                       | 85/344 [00:31<01:34,  2.74it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 86/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▊                       | 87/344 [00:31<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                       | 88/344 [00:32<01:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 89/344 [00:32<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  26%|████████                       | 90/344 [00:33<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|████████▏                      | 91/344 [00:33<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▎                      | 92/344 [00:33<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 93/344 [00:34<01:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▍                      | 94/344 [00:34<01:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                      | 95/344 [00:34<01:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 96/344 [00:35<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▋                      | 97/344 [00:35<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▊                      | 98/344 [00:35<01:31,  2.68it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▉                      | 99/344 [00:36<01:31,  2.67it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 100/344 [00:36<01:30,  2.69it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 101/344 [00:37<01:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 102/344 [00:37<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 103/344 [00:37<01:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 104/344 [00:38<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 105/344 [00:38<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 106/344 [00:38<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 107/344 [00:39<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 108/344 [00:39<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 109/344 [00:40<01:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 110/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 111/344 [00:40<01:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 112/344 [00:41<01:24,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 113/344 [00:41<01:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 114/344 [00:41<01:23,  2.74it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 115/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 116/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 117/344 [00:42<01:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 118/344 [00:43<01:22,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 119/344 [00:43<01:21,  2.75it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 120/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 121/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▋                   | 122/344 [00:44<01:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 123/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 124/344 [00:45<01:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 125/344 [00:45<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 126/344 [00:46<01:19,  2.74it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 127/344 [00:46<01:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 128/344 [00:47<01:20,  2.67it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 129/344 [00:47<01:19,  2.70it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 130/344 [00:47<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 131/344 [00:48<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 132/344 [00:48<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 133/344 [00:48<01:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 134/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 135/344 [00:49<01:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 136/344 [00:49<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 137/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 138/344 [00:50<01:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 139/344 [00:51<01:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 140/344 [00:51<01:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 141/344 [00:51<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 142/344 [00:52<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 143/344 [00:52<01:13,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 144/344 [00:52<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 145/344 [00:53<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 146/344 [00:53<01:12,  2.74it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 147/344 [00:53<01:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 148/344 [00:54<01:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 149/344 [00:54<01:11,  2.74it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 150/344 [00:55<01:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 151/344 [00:55<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 152/344 [00:55<01:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 153/344 [00:56<01:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 154/344 [00:56<01:10,  2.71it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  45%|█████████████▌                | 155/344 [00:56<01:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 156/344 [00:57<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 157/344 [00:57<01:09,  2.70it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 158/344 [00:57<01:09,  2.68it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 159/344 [00:58<01:08,  2.70it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 160/344 [00:58<01:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 161/344 [00:59<01:08,  2.69it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 162/344 [00:59<01:08,  2.67it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 163/344 [00:59<01:07,  2.69it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 164/344 [01:00<01:06,  2.69it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 165/344 [01:00<01:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 166/344 [01:00<01:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 167/344 [01:01<01:05,  2.69it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 168/344 [01:01<01:05,  2.67it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 169/344 [01:02<01:05,  2.68it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 170/344 [01:02<01:04,  2.69it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 171/344 [01:02<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 172/344 [01:03<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 173/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 174/344 [01:03<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 175/344 [01:04<01:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 176/344 [01:04<01:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 177/344 [01:04<01:00,  2.75it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 178/344 [01:05<01:00,  2.75it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 179/344 [01:05<01:00,  2.74it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 180/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 181/344 [01:06<00:59,  2.74it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 182/344 [01:06<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 183/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 184/344 [01:07<00:58,  2.74it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 185/344 [01:07<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 186/344 [01:08<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 187/344 [01:08<00:58,  2.70it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 188/344 [01:09<00:58,  2.66it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 189/344 [01:09<00:58,  2.66it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 190/344 [01:09<00:57,  2.68it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 191/344 [01:10<00:56,  2.70it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 192/344 [01:10<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 193/344 [01:10<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 194/344 [01:11<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 195/344 [01:11<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 196/344 [01:11<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 197/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 198/344 [01:12<00:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 199/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 200/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 201/344 [01:13<00:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 202/344 [01:14<00:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 203/344 [01:14<00:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 204/344 [01:14<00:51,  2.74it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 205/344 [01:15<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 206/344 [01:15<00:50,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 207/344 [01:16<00:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 208/344 [01:16<00:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 209/344 [01:16<00:49,  2.74it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 210/344 [01:17<00:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 211/344 [01:17<00:48,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 212/344 [01:17<00:48,  2.75it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 213/344 [01:18<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 214/344 [01:18<00:47,  2.74it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▊           | 215/344 [01:18<00:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 216/344 [01:19<00:46,  2.74it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 217/344 [01:19<00:47,  2.68it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 218/344 [01:20<00:47,  2.67it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 219/344 [01:20<00:46,  2.66it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 220/344 [01:20<00:46,  2.68it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 221/344 [01:21<00:45,  2.69it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▎          | 222/344 [01:21<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 223/344 [01:21<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 224/344 [01:22<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 225/344 [01:22<00:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 226/344 [01:22<00:43,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 227/344 [01:23<00:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 228/344 [01:23<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 229/344 [01:24<00:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 230/344 [01:24<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 231/344 [01:24<00:41,  2.74it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 232/344 [01:25<00:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 233/344 [01:25<00:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 234/344 [01:25<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 235/344 [01:26<00:39,  2.73it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 236/344 [01:26<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 237/344 [01:27<00:39,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 238/344 [01:27<00:38,  2.74it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 239/344 [01:27<00:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 240/344 [01:28<00:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 241/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 242/344 [01:28<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 243/344 [01:29<00:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 244/344 [01:29<00:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 245/344 [01:29<00:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 246/344 [01:30<00:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 247/344 [01:30<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 248/344 [01:31<00:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 249/344 [01:31<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 250/344 [01:31<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 251/344 [01:32<00:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 252/344 [01:32<00:33,  2.74it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 253/344 [01:32<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 254/344 [01:33<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 255/344 [01:33<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 256/344 [01:33<00:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 257/344 [01:34<00:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 258/344 [01:34<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 259/344 [01:35<00:31,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 260/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 261/344 [01:35<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 262/344 [01:36<00:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 263/344 [01:36<00:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 264/344 [01:36<00:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 265/344 [01:37<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 266/344 [01:37<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 267/344 [01:38<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 268/344 [01:38<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 269/344 [01:38<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 270/344 [01:39<00:27,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 271/344 [01:39<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 272/344 [01:39<00:26,  2.74it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 273/344 [01:40<00:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 274/344 [01:40<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 275/344 [01:40<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 276/344 [01:41<00:24,  2.74it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 277/344 [01:41<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 278/344 [01:42<00:24,  2.70it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 279/344 [01:42<00:24,  2.68it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 280/344 [01:42<00:23,  2.69it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 281/344 [01:43<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 282/344 [01:43<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 283/344 [01:43<00:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 284/344 [01:44<00:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 285/344 [01:44<00:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 286/344 [01:45<00:21,  2.74it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 287/344 [01:45<00:20,  2.74it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 288/344 [01:45<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 289/344 [01:46<00:20,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 290/344 [01:46<00:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 291/344 [01:46<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 292/344 [01:47<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 293/344 [01:47<00:18,  2.73it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▋    | 294/344 [01:47<00:18,  2.74it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 295/344 [01:48<00:17,  2.74it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 296/344 [01:48<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 297/344 [01:49<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 298/344 [01:49<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 299/344 [01:49<00:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 300/344 [01:50<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 301/344 [01:50<00:15,  2.74it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 302/344 [01:50<00:15,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 303/344 [01:51<00:14,  2.73it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 304/344 [01:51<00:14,  2.74it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 305/344 [01:51<00:14,  2.75it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 306/344 [01:52<00:13,  2.75it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 307/344 [01:52<00:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 308/344 [01:53<00:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 309/344 [01:53<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 310/344 [01:53<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 311/344 [01:54<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 312/344 [01:54<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 313/344 [01:54<00:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 314/344 [01:55<00:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 315/344 [01:55<00:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 316/344 [01:56<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 317/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 318/344 [01:56<00:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 319/344 [01:57<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 320/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 321/344 [01:57<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 322/344 [01:58<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 323/344 [01:58<00:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 324/344 [01:58<00:07,  2.69it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 325/344 [01:59<00:07,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 326/344 [01:59<00:06,  2.67it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 327/344 [02:00<00:06,  2.69it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 328/344 [02:00<00:06,  2.65it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 329/344 [02:00<00:05,  2.67it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 330/344 [02:01<00:05,  2.69it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 331/344 [02:01<00:04,  2.70it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 332/344 [02:01<00:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 333/344 [02:02<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 334/344 [02:02<00:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 335/344 [02:03<00:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 336/344 [02:03<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 337/344 [02:03<00:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 338/344 [02:04<00:02,  2.57it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 339/344 [02:04<00:01,  2.59it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 340/344 [02:04<00:01,  2.63it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 341/344 [02:05<00:01,  2.66it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 342/344 [02:05<00:00,  2.67it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 343/344 [02:06<00:00,  2.69it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 344/344 [02:06<00:00,  2.72it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [04:12<00:00, 126.27s/it]\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 13697.83it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:47<00:00, 578.48it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 467745.05it/s\r\n",
      "Evaluating: 100%|█████████████████████████████| 430/430 [00:54<00:00,  7.84it/s]\r\n"
     ]
    }
   ],
   "source": [
    "if cross_validation:\n",
    "    for i in range(1, K+1):\n",
    "        train_file = \"split_\" + str(i) + \"/train.json\"\n",
    "        predict_file = \"original/train.json\"\n",
    "        run_script(train_file, predict_file, batch_size, lr, epochs, max_seq_len, doc_stride)\n",
    "        !mv \"results/predictions_.json\" \"results/predictions_\"$i\".json\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation\n",
    "Calculate train/valid scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_kg_hide-output": false
   },
   "outputs": [],
   "source": [
    "def jaccard(str1, str2): \n",
    "    a = set(str1.lower().split()) \n",
    "    b = set(str2.lower().split())\n",
    "    c = a.intersection(b)\n",
    "    return float(len(c)) / (len(a) + len(b) - len(c))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(splits, np_train, post_processing=False):\n",
    "    K = len(splits)\n",
    "    predictions = [json.load(open('results/predictions_' + str(i+1) + '.json', 'r')) for i in range(K)]\n",
    "\n",
    "    train_score = [{'neutral':[], 'positive':[], 'negative':[], 'total':[]} for _ in range(K+1)]\n",
    "    valid_score = [{'neutral':[], 'positive':[], 'negative':[], 'total':[]} for _ in range(K+1)]\n",
    "\n",
    "    for train_idx, line in enumerate(np_train):\n",
    "        text_id = line[0]\n",
    "        text = line[1]\n",
    "        answer = line[2]\n",
    "        sentiment = line[-1]\n",
    "\n",
    "        if type(text) != str:\n",
    "            continue\n",
    "\n",
    "        for i, prediction in enumerate(predictions):\n",
    "            if text_id not in prediction:\n",
    "                print('key error:', text_id)\n",
    "                continue\n",
    "            else:\n",
    "                if post_processing and (sentiment == 'neutral' or len(text.split()) < 4): # post-processing\n",
    "                    score = jaccard(answer, text)\n",
    "                else:\n",
    "                    score = jaccard(answer, prediction[text_id])\n",
    "\n",
    "                if train_idx in splits[i]['valid_idx']:\n",
    "                    valid_score[i][sentiment].append(score)\n",
    "                    valid_score[i]['total'].append(score)\n",
    "                    valid_score[K][sentiment].append(score)\n",
    "                    valid_score[K]['total'].append(score)\n",
    "\n",
    "                else:\n",
    "                    train_score[i][sentiment].append(score)\n",
    "                    train_score[i]['total'].append(score)\n",
    "                    train_score[K][sentiment].append(score)\n",
    "                    train_score[K]['total'].append(score)\n",
    "\n",
    "    for i, score_dict in enumerate([train_score, valid_score]):\n",
    "        if i == 0:\n",
    "            print('train score \\n')\n",
    "        else:\n",
    "            print('valid score \\n')\n",
    "        for j in range(K+1):\n",
    "            for sentiment in ['neutral', 'positive', 'negative', 'total']:\n",
    "                score = np.array(score_dict[j][sentiment])\n",
    "                if j < K:\n",
    "                    print('split', j+1)\n",
    "                else:\n",
    "                    print('all data')\n",
    "                print(sentiment + ' - ' + str(len(score)) + ' examples, average score: ' + str(score.mean()))\n",
    "            print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train score \n",
      "\n",
      "split 1\n",
      "neutral - 8846 examples, average score: 0.9695459627071343\n",
      "split 1\n",
      "positive - 6894 examples, average score: 0.5615043951474108\n",
      "split 1\n",
      "negative - 6249 examples, average score: 0.5794303550477854\n",
      "split 1\n",
      "total - 21989 examples, average score: 0.7307506105301365\n",
      "\n",
      "split 2\n",
      "neutral - 8874 examples, average score: 0.9700491933222442\n",
      "split 2\n",
      "positive - 6895 examples, average score: 0.5611708256409118\n",
      "split 2\n",
      "negative - 6219 examples, average score: 0.5782213161020319\n",
      "split 2\n",
      "total - 21988 examples, average score: 0.7310099940501281\n",
      "\n",
      "split 3\n",
      "neutral - 8925 examples, average score: 0.9689940947351157\n",
      "split 3\n",
      "positive - 6849 examples, average score: 0.5647490910059514\n",
      "split 3\n",
      "negative - 6214 examples, average score: 0.5805615281327627\n",
      "split 3\n",
      "total - 21988 examples, average score: 0.7333021718950179\n",
      "\n",
      "split 4\n",
      "neutral - 8907 examples, average score: 0.9700262516409257\n",
      "split 4\n",
      "positive - 6834 examples, average score: 0.5597554039874609\n",
      "split 4\n",
      "negative - 6247 examples, average score: 0.5782704742594218\n",
      "split 4\n",
      "total - 21988 examples, average score: 0.7312101103745061\n",
      "\n",
      "split 5\n",
      "neutral - 8917 examples, average score: 0.9692515899283645\n",
      "split 5\n",
      "positive - 6856 examples, average score: 0.5680575700071037\n",
      "split 5\n",
      "negative - 6215 examples, average score: 0.5864766004465465\n",
      "split 5\n",
      "total - 21988 examples, average score: 0.7359637620126984\n",
      "\n",
      "all data\n",
      "neutral - 44469 examples, average score: 0.9695727961932304\n",
      "all data\n",
      "positive - 34328 examples, average score: 0.5630453798082936\n",
      "all data\n",
      "negative - 31144 examples, average score: 0.5805880976696323\n",
      "all data\n",
      "total - 109941 examples, average score: 0.7324473143394994\n",
      "\n",
      "valid score \n",
      "\n",
      "split 1\n",
      "neutral - 2271 examples, average score: 0.9697014893755529\n",
      "split 1\n",
      "positive - 1688 examples, average score: 0.5018548646749774\n",
      "split 1\n",
      "negative - 1537 examples, average score: 0.49356352598908454\n",
      "split 1\n",
      "total - 5496 examples, average score: 0.6928548459585999\n",
      "\n",
      "split 2\n",
      "neutral - 2243 examples, average score: 0.9677124952909482\n",
      "split 2\n",
      "positive - 1687 examples, average score: 0.5011621317666454\n",
      "split 2\n",
      "negative - 1567 examples, average score: 0.5063892189840136\n",
      "split 2\n",
      "total - 5497 examples, average score: 0.6930237492042708\n",
      "\n",
      "split 3\n",
      "neutral - 2192 examples, average score: 0.9719540935074289\n",
      "split 3\n",
      "positive - 1733 examples, average score: 0.4989492937969602\n",
      "split 3\n",
      "negative - 1572 examples, average score: 0.4963990640173825\n",
      "split 3\n",
      "total - 5497 examples, average score: 0.6868367887490888\n",
      "\n",
      "split 4\n",
      "neutral - 2210 examples, average score: 0.9677700656622015\n",
      "split 4\n",
      "positive - 1748 examples, average score: 0.4972656491406815\n",
      "split 4\n",
      "negative - 1539 examples, average score: 0.49699575066206336\n",
      "split 4\n",
      "total - 5497 examples, average score: 0.6863504930107862\n",
      "\n",
      "split 5\n",
      "neutral - 2200 examples, average score: 0.9708996550399843\n",
      "split 5\n",
      "positive - 1726 examples, average score: 0.48573820104023135\n",
      "split 5\n",
      "negative - 1571 examples, average score: 0.49766882291597414\n",
      "split 5\n",
      "total - 5497 examples, average score: 0.6833183730915773\n",
      "\n",
      "all data\n",
      "neutral - 11116 examples, average score: 0.969597487268729\n",
      "all data\n",
      "positive - 8582 examples, average score: 0.49695585454499824\n",
      "all data\n",
      "negative - 7786 examples, average score: 0.49822406310023226\n",
      "all data\n",
      "total - 27484 examples, average score: 0.6884766907103324\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if cross_validation:\n",
    "    evaluate(splits, np_train, post_processing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test\n",
    "Finetune a model for the test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_kg_hide-output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-04-05 09:26:28.303148: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\r\n",
      "100%|██████████████████████████████████| 27485/27485 [00:02<00:00, 12513.27it/s]\r\n",
      "convert squad examples to features: 100%|█| 27485/27485 [00:56<00:00, 486.41it/s\r\n",
      "add example index and unique id: 100%|█| 27485/27485 [00:00<00:00, 561245.78it/s\r\n",
      "Epoch:   0%|                                              | 0/2 [00:00<?, ?it/s]\r\n",
      "Iteration:   0%|                                        | 0/430 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/430 [00:00<04:25,  1.61it/s]\u001b[A\r\n",
      "Iteration:   0%|▏                               | 2/430 [00:00<03:53,  1.83it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 3/430 [00:01<03:29,  2.03it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/430 [00:01<03:13,  2.20it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 5/430 [00:02<03:02,  2.34it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 6/430 [00:02<02:53,  2.45it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 7/430 [00:02<02:47,  2.53it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 8/430 [00:03<02:43,  2.58it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 9/430 [00:03<02:40,  2.62it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                              | 10/430 [00:03<02:38,  2.65it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                              | 11/430 [00:04<02:36,  2.68it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                              | 12/430 [00:04<02:35,  2.69it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 13/430 [00:05<02:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 14/430 [00:05<02:34,  2.69it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 15/430 [00:05<02:33,  2.70it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 16/430 [00:06<02:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 17/430 [00:06<02:32,  2.70it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 18/430 [00:06<02:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 19/430 [00:07<02:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 20/430 [00:07<02:44,  2.50it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 21/430 [00:08<02:39,  2.56it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 22/430 [00:08<02:36,  2.60it/s]\u001b[A\r\n",
      "Iteration:   5%|█▋                             | 23/430 [00:08<02:34,  2.64it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 24/430 [00:09<02:32,  2.66it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 25/430 [00:09<02:30,  2.68it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 26/430 [00:09<02:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 27/430 [00:10<02:29,  2.70it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 28/430 [00:10<02:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 29/430 [00:11<02:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 30/430 [00:11<02:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 31/430 [00:11<02:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 32/430 [00:12<02:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 33/430 [00:12<02:25,  2.74it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 34/430 [00:12<02:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 35/430 [00:13<02:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 36/430 [00:13<02:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 37/430 [00:13<02:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 38/430 [00:14<02:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 39/430 [00:14<02:23,  2.73it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 40/430 [00:15<02:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 41/430 [00:15<02:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 42/430 [00:15<02:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 43/430 [00:16<02:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 44/430 [00:16<02:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 45/430 [00:16<02:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 46/430 [00:17<02:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 47/430 [00:17<02:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 48/430 [00:17<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 49/430 [00:18<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 50/430 [00:18<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 51/430 [00:19<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 52/430 [00:19<02:20,  2.69it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 53/430 [00:19<02:19,  2.70it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 54/430 [00:20<02:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 55/430 [00:20<02:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 56/430 [00:20<02:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 57/430 [00:21<02:16,  2.73it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 58/430 [00:21<02:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 59/430 [00:22<02:17,  2.70it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 60/430 [00:22<02:18,  2.68it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 61/430 [00:22<02:17,  2.68it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 62/430 [00:23<02:20,  2.62it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 63/430 [00:23<02:18,  2.65it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 64/430 [00:23<02:16,  2.67it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 65/430 [00:24<02:15,  2.69it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 66/430 [00:24<02:14,  2.70it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 67/430 [00:25<02:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 68/430 [00:25<02:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 69/430 [00:25<02:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 70/430 [00:26<02:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████                          | 71/430 [00:26<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 72/430 [00:26<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 73/430 [00:27<02:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 74/430 [00:27<02:12,  2.69it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 75/430 [00:28<02:11,  2.69it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 76/430 [00:28<02:11,  2.70it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 77/430 [00:28<02:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 78/430 [00:29<02:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 79/430 [00:29<02:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 80/430 [00:29<02:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 81/430 [00:30<02:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 82/430 [00:30<02:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 83/430 [00:30<02:06,  2.74it/s]\u001b[A\r\n",
      "Iteration:  20%|██████                         | 84/430 [00:31<02:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 85/430 [00:31<02:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 86/430 [00:32<02:07,  2.69it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 87/430 [00:32<02:08,  2.68it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 88/430 [00:32<02:07,  2.69it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 89/430 [00:33<02:06,  2.70it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 90/430 [00:33<02:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 91/430 [00:33<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▋                        | 92/430 [00:34<02:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 93/430 [00:34<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 94/430 [00:34<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 95/430 [00:35<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 96/430 [00:35<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|██████▉                        | 97/430 [00:36<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 98/430 [00:36<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 99/430 [00:36<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|██████▉                       | 100/430 [00:37<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                       | 101/430 [00:37<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████                       | 102/430 [00:37<02:00,  2.73it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▏                      | 103/430 [00:38<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                      | 104/430 [00:38<02:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                      | 105/430 [00:39<01:59,  2.71it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▍                      | 106/430 [00:39<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▍                      | 107/430 [00:39<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▌                      | 108/430 [00:40<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▌                      | 109/430 [00:40<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▋                      | 110/430 [00:40<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▋                      | 111/430 [00:41<01:58,  2.69it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▊                      | 112/430 [00:41<01:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                      | 113/430 [00:41<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|███████▉                      | 114/430 [00:42<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|████████                      | 115/430 [00:42<01:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████                      | 116/430 [00:43<01:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▏                     | 117/430 [00:43<01:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▏                     | 118/430 [00:43<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▎                     | 119/430 [00:44<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▎                     | 120/430 [00:44<01:53,  2.74it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▍                     | 121/430 [00:44<01:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                     | 122/430 [00:45<01:52,  2.74it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▌                     | 123/430 [00:45<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 124/430 [00:46<01:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 125/430 [00:46<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 126/430 [00:46<01:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▊                     | 127/430 [00:47<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 128/430 [00:47<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 129/430 [00:47<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 130/430 [00:48<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████▏                    | 131/430 [00:48<01:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 132/430 [00:48<01:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 133/430 [00:49<01:49,  2.72it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 134/430 [00:49<01:48,  2.73it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 135/430 [00:50<01:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▍                    | 136/430 [00:50<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 137/430 [00:50<01:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 138/430 [00:51<01:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 139/430 [00:51<01:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 140/430 [00:51<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 141/430 [00:52<01:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 142/430 [00:52<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 143/430 [00:52<01:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 144/430 [00:53<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 145/430 [00:53<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 146/430 [00:54<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 147/430 [00:54<01:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 148/430 [00:54<01:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 149/430 [00:55<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 150/430 [00:55<01:43,  2.70it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 151/430 [00:55<01:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 152/430 [00:56<01:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 153/430 [00:56<01:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 154/430 [00:57<01:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 155/430 [00:57<01:41,  2.71it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 156/430 [00:57<01:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 157/430 [00:58<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 158/430 [00:58<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 159/430 [00:58<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 160/430 [00:59<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 161/430 [00:59<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 162/430 [00:59<01:38,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 163/430 [01:00<01:37,  2.73it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 164/430 [01:00<01:37,  2.74it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 165/430 [01:01<01:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 166/430 [01:01<01:36,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 167/430 [01:01<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 168/430 [01:02<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 169/430 [01:02<01:35,  2.74it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 170/430 [01:02<01:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 171/430 [01:03<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 172/430 [01:03<01:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 173/430 [01:04<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████▏                 | 174/430 [01:04<01:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 175/430 [01:04<01:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 176/430 [01:05<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 177/430 [01:05<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 178/430 [01:05<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 179/430 [01:06<01:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 180/430 [01:06<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 181/430 [01:06<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 182/430 [01:07<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 183/430 [01:07<01:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 184/430 [01:08<01:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 185/430 [01:08<01:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 186/430 [01:08<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  43%|█████████████                 | 187/430 [01:09<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 188/430 [01:09<01:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 189/430 [01:09<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 190/430 [01:10<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 191/430 [01:10<01:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 192/430 [01:10<01:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 193/430 [01:11<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 194/430 [01:11<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 195/430 [01:12<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 196/430 [01:12<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 197/430 [01:12<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 198/430 [01:13<01:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▉                | 199/430 [01:13<01:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 200/430 [01:13<01:25,  2.70it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 201/430 [01:14<01:24,  2.70it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 202/430 [01:14<01:24,  2.70it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 203/430 [01:15<01:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 204/430 [01:15<01:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 205/430 [01:15<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 206/430 [01:16<01:22,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 207/430 [01:16<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▌               | 208/430 [01:16<01:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 209/430 [01:17<01:21,  2.73it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 210/430 [01:17<01:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 211/430 [01:17<01:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 212/430 [01:18<01:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▊               | 213/430 [01:18<01:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 214/430 [01:19<01:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 215/430 [01:19<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 216/430 [01:19<01:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████▏              | 217/430 [01:20<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 218/430 [01:20<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 219/430 [01:20<01:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 220/430 [01:21<01:17,  2.70it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 221/430 [01:21<01:17,  2.69it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▍              | 222/430 [01:22<01:18,  2.66it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 223/430 [01:22<01:18,  2.63it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 224/430 [01:22<01:18,  2.63it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 225/430 [01:23<01:17,  2.66it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 226/430 [01:23<01:16,  2.68it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 227/430 [01:23<01:15,  2.70it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 228/430 [01:24<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 229/430 [01:24<01:14,  2.70it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 230/430 [01:25<01:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████              | 231/430 [01:25<01:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 232/430 [01:25<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 233/430 [01:26<01:12,  2.73it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 234/430 [01:26<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 235/430 [01:26<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 236/430 [01:27<01:11,  2.73it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 237/430 [01:27<01:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 238/430 [01:28<01:11,  2.68it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 239/430 [01:28<01:10,  2.69it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 240/430 [01:28<01:10,  2.70it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 241/430 [01:29<01:09,  2.70it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 242/430 [01:29<01:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|████████████████▉             | 243/430 [01:29<01:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 244/430 [01:30<01:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 245/430 [01:30<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 246/430 [01:30<01:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 247/430 [01:31<01:07,  2.70it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 248/430 [01:31<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 249/430 [01:32<01:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 250/430 [01:32<01:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 251/430 [01:32<01:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 252/430 [01:33<01:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 253/430 [01:33<01:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 254/430 [01:33<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 255/430 [01:34<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▊            | 256/430 [01:34<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 257/430 [01:34<01:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 258/430 [01:35<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 259/430 [01:35<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 260/430 [01:36<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 261/430 [01:36<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 262/430 [01:36<01:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 263/430 [01:37<01:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 264/430 [01:37<01:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 265/430 [01:37<01:00,  2.71it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 266/430 [01:38<01:00,  2.70it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 267/430 [01:38<01:00,  2.70it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 268/430 [01:39<00:59,  2.71it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 269/430 [01:39<00:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 270/430 [01:39<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 271/430 [01:40<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 272/430 [01:40<00:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 273/430 [01:40<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 274/430 [01:41<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 275/430 [01:41<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 276/430 [01:41<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 277/430 [01:42<00:56,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 278/430 [01:42<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 279/430 [01:43<00:55,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 280/430 [01:43<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 281/430 [01:43<00:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 282/430 [01:44<00:54,  2.72it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 283/430 [01:44<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 284/430 [01:44<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 285/430 [01:45<00:53,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 286/430 [01:45<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 287/430 [01:46<00:52,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 288/430 [01:46<00:51,  2.73it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 289/430 [01:46<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 290/430 [01:47<00:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 291/430 [01:47<00:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 292/430 [01:47<00:51,  2.66it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 293/430 [01:48<00:51,  2.67it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▌         | 294/430 [01:48<00:50,  2.68it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 295/430 [01:49<00:50,  2.69it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 296/430 [01:49<00:49,  2.69it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 297/430 [01:49<00:49,  2.69it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 298/430 [01:50<00:48,  2.70it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▊         | 299/430 [01:50<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 300/430 [01:50<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 301/430 [01:51<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 302/430 [01:51<00:47,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████▏        | 303/430 [01:51<00:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 304/430 [01:52<00:46,  2.72it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 305/430 [01:52<00:46,  2.70it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 306/430 [01:53<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▍        | 307/430 [01:53<00:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 308/430 [01:53<00:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 309/430 [01:54<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 310/430 [01:54<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 311/430 [01:54<00:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 312/430 [01:55<00:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 313/430 [01:55<00:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 314/430 [01:56<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 315/430 [01:56<00:42,  2.72it/s]\u001b[A\r\n",
      "Iteration:  73%|██████████████████████        | 316/430 [01:56<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 317/430 [01:57<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 318/430 [01:57<00:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 319/430 [01:57<00:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 320/430 [01:58<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 321/430 [01:58<00:40,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 322/430 [01:58<00:40,  2.67it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 323/430 [01:59<00:40,  2.66it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 324/430 [01:59<00:39,  2.68it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 325/430 [02:00<00:39,  2.69it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 326/430 [02:00<00:38,  2.70it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 327/430 [02:00<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 328/430 [02:01<00:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|██████████████████████▉       | 329/430 [02:01<00:37,  2.69it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 330/430 [02:01<00:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 331/430 [02:02<00:36,  2.70it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 332/430 [02:02<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 333/430 [02:03<00:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 334/430 [02:03<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 335/430 [02:03<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 336/430 [02:04<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 337/430 [02:04<00:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▌      | 338/430 [02:04<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 339/430 [02:05<00:33,  2.72it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 340/430 [02:05<00:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 341/430 [02:05<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▊      | 342/430 [02:06<00:32,  2.73it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 343/430 [02:06<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 344/430 [02:07<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 345/430 [02:07<00:31,  2.72it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████▏     | 346/430 [02:07<00:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 347/430 [02:08<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 348/430 [02:08<00:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 349/430 [02:08<00:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 350/430 [02:09<00:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▍     | 351/430 [02:09<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 352/430 [02:10<00:28,  2.73it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 353/430 [02:10<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 354/430 [02:10<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 355/430 [02:11<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 356/430 [02:11<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 357/430 [02:11<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 358/430 [02:12<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 359/430 [02:12<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 360/430 [02:12<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 361/430 [02:13<00:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 362/430 [02:13<00:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 363/430 [02:14<00:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 364/430 [02:14<00:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 365/430 [02:14<00:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 366/430 [02:15<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 367/430 [02:15<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 368/430 [02:15<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 369/430 [02:16<00:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 370/430 [02:16<00:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 371/430 [02:16<00:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 372/430 [02:17<00:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 373/430 [02:17<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 374/430 [02:18<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 375/430 [02:18<00:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 376/430 [02:18<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 377/430 [02:19<00:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 378/430 [02:19<00:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 379/430 [02:19<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 380/430 [02:20<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 381/430 [02:20<00:18,  2.70it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 382/430 [02:21<00:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 383/430 [02:21<00:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 384/430 [02:21<00:18,  2.55it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 385/430 [02:22<00:17,  2.56it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 386/430 [02:22<00:16,  2.60it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 387/430 [02:22<00:16,  2.64it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 388/430 [02:23<00:15,  2.63it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████▏  | 389/430 [02:23<00:15,  2.65it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 390/430 [02:24<00:15,  2.65it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 391/430 [02:24<00:14,  2.65it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 392/430 [02:24<00:14,  2.67it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 393/430 [02:25<00:13,  2.68it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 394/430 [02:25<00:13,  2.67it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 395/430 [02:25<00:13,  2.68it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 396/430 [02:26<00:12,  2.69it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 397/430 [02:26<00:12,  2.70it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 398/430 [02:27<00:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 399/430 [02:27<00:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 400/430 [02:27<00:11,  2.68it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 401/430 [02:28<00:10,  2.70it/s]\u001b[A\r\n",
      "Iteration:  93%|████████████████████████████  | 402/430 [02:28<00:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 403/430 [02:28<00:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 404/430 [02:29<00:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 405/430 [02:29<00:09,  2.69it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 406/430 [02:30<00:08,  2.70it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 407/430 [02:30<00:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 408/430 [02:30<00:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 409/430 [02:31<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 410/430 [02:31<00:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 411/430 [02:31<00:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 412/430 [02:32<00:06,  2.70it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 413/430 [02:32<00:06,  2.71it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▉ | 414/430 [02:33<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 415/430 [02:33<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 416/430 [02:33<00:05,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 417/430 [02:34<00:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 418/430 [02:34<00:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 419/430 [02:34<00:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 420/430 [02:35<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 421/430 [02:35<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 422/430 [02:35<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▌| 423/430 [02:36<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 424/430 [02:36<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 425/430 [02:37<00:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 426/430 [02:37<00:01,  2.74it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 427/430 [02:37<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▊| 428/430 [02:38<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 429/430 [02:38<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 430/430 [02:38<00:00,  2.71it/s]\r\n",
      "Epoch:  50%|██████████████████▌                  | 1/2 [02:38<02:38, 158.70s/it]\r\n",
      "Iteration:   0%|                                        | 0/430 [00:00<?, ?it/s]\u001b[A\r\n",
      "Iteration:   0%|                                | 1/430 [00:00<02:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:   0%|▏                               | 2/430 [00:00<02:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▏                               | 3/430 [00:01<02:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 4/430 [00:01<02:36,  2.73it/s]\u001b[A\r\n",
      "Iteration:   1%|▎                               | 5/430 [00:01<02:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:   1%|▍                               | 6/430 [00:02<02:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 7/430 [00:02<02:35,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▌                               | 8/430 [00:02<02:34,  2.73it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                               | 9/430 [00:03<02:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:   2%|▋                              | 10/430 [00:03<02:33,  2.73it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                              | 11/430 [00:04<02:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:   3%|▊                              | 12/430 [00:04<02:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:   3%|▉                              | 13/430 [00:04<02:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 14/430 [00:05<02:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:   3%|█                              | 15/430 [00:05<02:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 16/430 [00:05<02:32,  2.72it/s]\u001b[A\r\n",
      "Iteration:   4%|█▏                             | 17/430 [00:06<02:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 18/430 [00:06<02:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:   4%|█▎                             | 19/430 [00:06<02:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▍                             | 20/430 [00:07<02:30,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 21/430 [00:07<02:29,  2.73it/s]\u001b[A\r\n",
      "Iteration:   5%|█▌                             | 22/430 [00:08<02:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:   5%|█▋                             | 23/430 [00:08<02:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▋                             | 24/430 [00:08<02:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 25/430 [00:09<02:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▊                             | 26/430 [00:09<02:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:   6%|█▉                             | 27/430 [00:09<02:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 28/430 [00:10<02:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:   7%|██                             | 29/430 [00:10<02:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 30/430 [00:11<02:28,  2.70it/s]\u001b[A\r\n",
      "Iteration:   7%|██▏                            | 31/430 [00:11<02:27,  2.70it/s]\u001b[A\r\n",
      "Iteration:   7%|██▎                            | 32/430 [00:11<02:26,  2.71it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 33/430 [00:12<02:26,  2.71it/s]\u001b[A\r\n",
      "Iteration:   8%|██▍                            | 34/430 [00:12<02:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 35/430 [00:12<02:26,  2.70it/s]\u001b[A\r\n",
      "Iteration:   8%|██▌                            | 36/430 [00:13<02:25,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 37/430 [00:13<02:25,  2.70it/s]\u001b[A\r\n",
      "Iteration:   9%|██▋                            | 38/430 [00:13<02:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▊                            | 39/430 [00:14<02:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:   9%|██▉                            | 40/430 [00:14<02:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|██▉                            | 41/430 [00:15<02:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 42/430 [00:15<02:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  10%|███                            | 43/430 [00:15<02:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 44/430 [00:16<02:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  10%|███▏                           | 45/430 [00:16<02:21,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▎                           | 46/430 [00:16<02:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 47/430 [00:17<02:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▍                           | 48/430 [00:17<02:20,  2.72it/s]\u001b[A\r\n",
      "Iteration:  11%|███▌                           | 49/430 [00:18<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▌                           | 50/430 [00:18<02:19,  2.73it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 51/430 [00:18<02:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  12%|███▋                           | 52/430 [00:19<02:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  12%|███▊                           | 53/430 [00:19<02:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 54/430 [00:19<02:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  13%|███▉                           | 55/430 [00:20<02:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 56/430 [00:20<02:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████                           | 57/430 [00:20<02:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  13%|████▏                          | 58/430 [00:21<02:17,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 59/430 [00:21<02:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  14%|████▎                          | 60/430 [00:22<02:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 61/430 [00:22<02:16,  2.71it/s]\u001b[A\r\n",
      "Iteration:  14%|████▍                          | 62/430 [00:22<02:15,  2.71it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 63/430 [00:23<02:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▌                          | 64/430 [00:23<02:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▋                          | 65/430 [00:23<02:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  15%|████▊                          | 66/430 [00:24<02:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|████▊                          | 67/430 [00:24<02:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|████▉                          | 68/430 [00:25<02:13,  2.72it/s]\u001b[A/opt/conda/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:224: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\r\n",
      "  warnings.warn(\"To get the last learning rate computed by the scheduler, \"\r\n",
      "\r\n",
      "Iteration:  16%|████▉                          | 69/430 [00:25<02:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  16%|█████                          | 70/430 [00:25<02:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████                          | 71/430 [00:26<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▏                         | 72/430 [00:26<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 73/430 [00:26<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▎                         | 74/430 [00:27<02:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  17%|█████▍                         | 75/430 [00:27<02:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▍                         | 76/430 [00:27<02:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 77/430 [00:28<02:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▌                         | 78/430 [00:28<02:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  18%|█████▋                         | 79/430 [00:29<02:09,  2.71it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 80/430 [00:29<02:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▊                         | 81/430 [00:29<02:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 82/430 [00:30<02:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  19%|█████▉                         | 83/430 [00:30<02:07,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████                         | 84/430 [00:30<02:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 85/430 [00:31<02:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▏                        | 86/430 [00:31<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 87/430 [00:32<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  20%|██████▎                        | 88/430 [00:32<02:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 89/430 [00:32<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▍                        | 90/430 [00:33<02:04,  2.74it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▌                        | 91/430 [00:33<02:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  21%|██████▋                        | 92/430 [00:33<02:03,  2.74it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▋                        | 93/430 [00:34<02:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 94/430 [00:34<02:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▊                        | 95/430 [00:34<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  22%|██████▉                        | 96/430 [00:35<02:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  23%|██████▉                        | 97/430 [00:35<02:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                        | 98/430 [00:36<02:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  23%|███████▏                       | 99/430 [00:36<02:02,  2.71it/s]\u001b[A\r\n",
      "Iteration:  23%|██████▉                       | 100/430 [00:36<02:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  23%|███████                       | 101/430 [00:37<02:01,  2.71it/s]\u001b[A\r\n",
      "Iteration:  24%|███████                       | 102/430 [00:37<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▏                      | 103/430 [00:37<02:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                      | 104/430 [00:38<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  24%|███████▎                      | 105/430 [00:38<01:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▍                      | 106/430 [00:38<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▍                      | 107/430 [00:39<01:58,  2.72it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▌                      | 108/430 [00:39<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:  25%|███████▌                      | 109/430 [00:40<01:58,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▋                      | 110/430 [00:40<01:57,  2.71it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▋                      | 111/430 [00:40<01:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▊                      | 112/430 [00:41<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  26%|███████▉                      | 113/430 [00:41<01:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|███████▉                      | 114/430 [00:41<01:56,  2.71it/s]\u001b[A\r\n",
      "Iteration:  27%|████████                      | 115/430 [00:42<01:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████                      | 116/430 [00:42<01:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▏                     | 117/430 [00:43<01:54,  2.73it/s]\u001b[A\r\n",
      "Iteration:  27%|████████▏                     | 118/430 [00:43<01:55,  2.70it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▎                     | 119/430 [00:43<01:55,  2.68it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▎                     | 120/430 [00:44<01:55,  2.68it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▍                     | 121/430 [00:44<01:54,  2.69it/s]\u001b[A\r\n",
      "Iteration:  28%|████████▌                     | 122/430 [00:44<01:54,  2.69it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▌                     | 123/430 [00:45<01:53,  2.70it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 124/430 [00:45<01:53,  2.70it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▋                     | 125/430 [00:46<01:52,  2.70it/s]\u001b[A\r\n",
      "Iteration:  29%|████████▊                     | 126/430 [00:46<01:51,  2.72it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▊                     | 127/430 [00:46<01:51,  2.71it/s]\u001b[A\r\n",
      "Iteration:  30%|████████▉                     | 128/430 [00:47<01:52,  2.69it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 129/430 [00:47<01:51,  2.69it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████                     | 130/430 [00:47<01:51,  2.70it/s]\u001b[A\r\n",
      "Iteration:  30%|█████████▏                    | 131/430 [00:48<01:50,  2.71it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▏                    | 132/430 [00:48<01:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 133/430 [00:48<01:50,  2.68it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▎                    | 134/430 [00:49<01:50,  2.68it/s]\u001b[A\r\n",
      "Iteration:  31%|█████████▍                    | 135/430 [00:49<01:50,  2.67it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▍                    | 136/430 [00:50<01:50,  2.67it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▌                    | 137/430 [00:50<01:49,  2.69it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 138/430 [00:50<01:48,  2.69it/s]\u001b[A\r\n",
      "Iteration:  32%|█████████▋                    | 139/430 [00:51<01:47,  2.70it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 140/430 [00:51<01:47,  2.70it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▊                    | 141/430 [00:51<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 142/430 [00:52<01:46,  2.71it/s]\u001b[A\r\n",
      "Iteration:  33%|█████████▉                    | 143/430 [00:52<01:45,  2.72it/s]\u001b[A\r\n",
      "Iteration:  33%|██████████                    | 144/430 [00:53<01:45,  2.71it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████                    | 145/430 [00:53<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▏                   | 146/430 [00:53<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 147/430 [00:54<01:44,  2.72it/s]\u001b[A\r\n",
      "Iteration:  34%|██████████▎                   | 148/430 [00:54<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 149/430 [00:54<01:43,  2.72it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▍                   | 150/430 [00:55<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 151/430 [00:55<01:42,  2.73it/s]\u001b[A\r\n",
      "Iteration:  35%|██████████▌                   | 152/430 [00:55<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 153/430 [00:56<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▋                   | 154/430 [00:56<01:41,  2.73it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▊                   | 155/430 [00:57<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  36%|██████████▉                   | 156/430 [00:57<01:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|██████████▉                   | 157/430 [00:57<01:40,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 158/430 [00:58<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████                   | 159/430 [00:58<01:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 160/430 [00:58<01:39,  2.73it/s]\u001b[A\r\n",
      "Iteration:  37%|███████████▏                  | 161/430 [00:59<01:38,  2.72it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 162/430 [00:59<01:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▎                  | 163/430 [01:00<01:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▍                  | 164/430 [01:00<01:38,  2.70it/s]\u001b[A\r\n",
      "Iteration:  38%|███████████▌                  | 165/430 [01:00<01:37,  2.70it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▌                  | 166/430 [01:01<01:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 167/430 [01:01<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▋                  | 168/430 [01:01<01:36,  2.72it/s]\u001b[A\r\n",
      "Iteration:  39%|███████████▊                  | 169/430 [01:02<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▊                  | 170/430 [01:02<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|███████████▉                  | 171/430 [01:02<01:35,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 172/430 [01:03<01:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████                  | 173/430 [01:03<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  40%|████████████▏                 | 174/430 [01:04<01:34,  2.72it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▏                 | 175/430 [01:04<01:34,  2.71it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 176/430 [01:04<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▎                 | 177/430 [01:05<01:33,  2.70it/s]\u001b[A\r\n",
      "Iteration:  41%|████████████▍                 | 178/430 [01:05<01:33,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▍                 | 179/430 [01:05<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▌                 | 180/430 [01:06<01:32,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 181/430 [01:06<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  42%|████████████▋                 | 182/430 [01:07<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 183/430 [01:07<01:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▊                 | 184/430 [01:07<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 185/430 [01:08<01:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|████████████▉                 | 186/430 [01:08<01:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  43%|█████████████                 | 187/430 [01:08<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████                 | 188/430 [01:09<01:29,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▏                | 189/430 [01:09<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 190/430 [01:09<01:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  44%|█████████████▎                | 191/430 [01:10<01:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 192/430 [01:10<01:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▍                | 193/430 [01:11<01:27,  2.71it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 194/430 [01:11<01:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  45%|█████████████▌                | 195/430 [01:11<01:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 196/430 [01:12<01:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▋                | 197/430 [01:12<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▊                | 198/430 [01:12<01:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  46%|█████████████▉                | 199/430 [01:13<01:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|█████████████▉                | 200/430 [01:13<01:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 201/430 [01:14<01:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████                | 202/430 [01:14<01:24,  2.71it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 203/430 [01:14<01:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  47%|██████████████▏               | 204/430 [01:15<01:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 205/430 [01:15<01:23,  2.70it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▎               | 206/430 [01:15<01:22,  2.71it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▍               | 207/430 [01:16<01:22,  2.72it/s]\u001b[A\r\n",
      "Iteration:  48%|██████████████▌               | 208/430 [01:16<01:21,  2.71it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▌               | 209/430 [01:16<01:21,  2.71it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 210/430 [01:17<01:21,  2.70it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▋               | 211/430 [01:17<01:21,  2.70it/s]\u001b[A\r\n",
      "Iteration:  49%|██████████████▊               | 212/430 [01:18<01:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▊               | 213/430 [01:18<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|██████████████▉               | 214/430 [01:18<01:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 215/430 [01:19<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████               | 216/430 [01:19<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  50%|███████████████▏              | 217/430 [01:19<01:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▏              | 218/430 [01:20<01:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 219/430 [01:20<01:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▎              | 220/430 [01:21<01:17,  2.70it/s]\u001b[A\r\n",
      "Iteration:  51%|███████████████▍              | 221/430 [01:21<01:18,  2.66it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▍              | 222/430 [01:21<01:17,  2.68it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▌              | 223/430 [01:22<01:16,  2.70it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 224/430 [01:22<01:16,  2.69it/s]\u001b[A\r\n",
      "Iteration:  52%|███████████████▋              | 225/430 [01:22<01:16,  2.70it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 226/430 [01:23<01:15,  2.70it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▊              | 227/430 [01:23<01:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 228/430 [01:23<01:14,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|███████████████▉              | 229/430 [01:24<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  53%|████████████████              | 230/430 [01:24<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████              | 231/430 [01:25<01:13,  2.72it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▏             | 232/430 [01:25<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 233/430 [01:25<01:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  54%|████████████████▎             | 234/430 [01:26<01:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 235/430 [01:26<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▍             | 236/430 [01:26<01:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 237/430 [01:27<01:11,  2.71it/s]\u001b[A\r\n",
      "Iteration:  55%|████████████████▌             | 238/430 [01:27<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 239/430 [01:28<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▋             | 240/430 [01:28<01:10,  2.71it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▊             | 241/430 [01:28<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  56%|████████████████▉             | 242/430 [01:29<01:09,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|████████████████▉             | 243/430 [01:29<01:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 244/430 [01:29<01:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████             | 245/430 [01:30<01:08,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 246/430 [01:30<01:07,  2.71it/s]\u001b[A\r\n",
      "Iteration:  57%|█████████████████▏            | 247/430 [01:30<01:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 248/430 [01:31<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▎            | 249/430 [01:31<01:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▍            | 250/430 [01:32<01:06,  2.69it/s]\u001b[A\r\n",
      "Iteration:  58%|█████████████████▌            | 251/430 [01:32<01:07,  2.66it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▌            | 252/430 [01:32<01:06,  2.68it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 253/430 [01:33<01:05,  2.69it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▋            | 254/430 [01:33<01:05,  2.70it/s]\u001b[A\r\n",
      "Iteration:  59%|█████████████████▊            | 255/430 [01:33<01:04,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▊            | 256/430 [01:34<01:04,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|█████████████████▉            | 257/430 [01:34<01:03,  2.71it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 258/430 [01:35<01:03,  2.72it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████            | 259/430 [01:35<01:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  60%|██████████████████▏           | 260/430 [01:35<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▏           | 261/430 [01:36<01:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 262/430 [01:36<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▎           | 263/430 [01:36<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  61%|██████████████████▍           | 264/430 [01:37<01:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▍           | 265/430 [01:37<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▌           | 266/430 [01:38<01:00,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 267/430 [01:38<00:59,  2.72it/s]\u001b[A\r\n",
      "Iteration:  62%|██████████████████▋           | 268/430 [01:38<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 269/430 [01:39<00:59,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▊           | 270/430 [01:39<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 271/430 [01:39<00:58,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|██████████████████▉           | 272/430 [01:40<00:57,  2.73it/s]\u001b[A\r\n",
      "Iteration:  63%|███████████████████           | 273/430 [01:40<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████           | 274/430 [01:40<00:57,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▏          | 275/430 [01:41<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 276/430 [01:41<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  64%|███████████████████▎          | 277/430 [01:42<00:56,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 278/430 [01:42<00:55,  2.72it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▍          | 279/430 [01:42<00:55,  2.71it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 280/430 [01:43<00:59,  2.54it/s]\u001b[A\r\n",
      "Iteration:  65%|███████████████████▌          | 281/430 [01:43<00:58,  2.54it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 282/430 [01:43<00:57,  2.58it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▋          | 283/430 [01:44<00:56,  2.62it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▊          | 284/430 [01:44<00:55,  2.65it/s]\u001b[A\r\n",
      "Iteration:  66%|███████████████████▉          | 285/430 [01:45<00:54,  2.67it/s]\u001b[A\r\n",
      "Iteration:  67%|███████████████████▉          | 286/430 [01:45<00:54,  2.66it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 287/430 [01:45<00:54,  2.63it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████          | 288/430 [01:46<00:53,  2.65it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 289/430 [01:46<00:52,  2.67it/s]\u001b[A\r\n",
      "Iteration:  67%|████████████████████▏         | 290/430 [01:46<00:52,  2.67it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 291/430 [01:47<00:51,  2.69it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▎         | 292/430 [01:47<00:51,  2.69it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▍         | 293/430 [01:48<00:50,  2.69it/s]\u001b[A\r\n",
      "Iteration:  68%|████████████████████▌         | 294/430 [01:48<00:50,  2.70it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▌         | 295/430 [01:48<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 296/430 [01:49<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▋         | 297/430 [01:49<00:49,  2.71it/s]\u001b[A\r\n",
      "Iteration:  69%|████████████████████▊         | 298/430 [01:49<00:48,  2.71it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▊         | 299/430 [01:50<00:48,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|████████████████████▉         | 300/430 [01:50<00:47,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 301/430 [01:51<00:47,  2.72it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████         | 302/430 [01:51<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  70%|█████████████████████▏        | 303/430 [01:51<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▏        | 304/430 [01:52<00:46,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 305/430 [01:52<00:45,  2.73it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▎        | 306/430 [01:52<00:46,  2.69it/s]\u001b[A\r\n",
      "Iteration:  71%|█████████████████████▍        | 307/430 [01:53<00:45,  2.68it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▍        | 308/430 [01:53<00:45,  2.69it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▌        | 309/430 [01:53<00:44,  2.70it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 310/430 [01:54<00:44,  2.71it/s]\u001b[A\r\n",
      "Iteration:  72%|█████████████████████▋        | 311/430 [01:54<00:44,  2.70it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 312/430 [01:55<00:43,  2.69it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▊        | 313/430 [01:55<00:43,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 314/430 [01:55<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|█████████████████████▉        | 315/430 [01:56<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  73%|██████████████████████        | 316/430 [01:56<00:42,  2.71it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████        | 317/430 [01:56<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▏       | 318/430 [01:57<00:41,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 319/430 [01:57<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  74%|██████████████████████▎       | 320/430 [01:58<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 321/430 [01:58<00:40,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▍       | 322/430 [01:58<00:39,  2.72it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 323/430 [01:59<00:39,  2.71it/s]\u001b[A\r\n",
      "Iteration:  75%|██████████████████████▌       | 324/430 [01:59<00:39,  2.70it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 325/430 [01:59<00:38,  2.70it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▋       | 326/430 [02:00<00:38,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▊       | 327/430 [02:00<00:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  76%|██████████████████████▉       | 328/430 [02:00<00:37,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|██████████████████████▉       | 329/430 [02:01<00:37,  2.72it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 330/430 [02:01<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████       | 331/430 [02:02<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 332/430 [02:02<00:36,  2.71it/s]\u001b[A\r\n",
      "Iteration:  77%|███████████████████████▏      | 333/430 [02:02<00:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 334/430 [02:03<00:35,  2.71it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▎      | 335/430 [02:03<00:35,  2.70it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▍      | 336/430 [02:03<00:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  78%|███████████████████████▌      | 337/430 [02:04<00:34,  2.70it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▌      | 338/430 [02:04<00:34,  2.63it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 339/430 [02:05<00:34,  2.64it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▋      | 340/430 [02:05<00:34,  2.63it/s]\u001b[A\r\n",
      "Iteration:  79%|███████████████████████▊      | 341/430 [02:05<00:33,  2.66it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▊      | 342/430 [02:06<00:32,  2.68it/s]\u001b[A\r\n",
      "Iteration:  80%|███████████████████████▉      | 343/430 [02:06<00:32,  2.69it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 344/430 [02:06<00:31,  2.70it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████      | 345/430 [02:07<00:31,  2.71it/s]\u001b[A\r\n",
      "Iteration:  80%|████████████████████████▏     | 346/430 [02:07<00:30,  2.72it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▏     | 347/430 [02:08<00:30,  2.70it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 348/430 [02:08<00:30,  2.71it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▎     | 349/430 [02:08<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  81%|████████████████████████▍     | 350/430 [02:09<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▍     | 351/430 [02:09<00:29,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▌     | 352/430 [02:09<00:28,  2.71it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 353/430 [02:10<00:28,  2.72it/s]\u001b[A\r\n",
      "Iteration:  82%|████████████████████████▋     | 354/430 [02:10<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 355/430 [02:10<00:27,  2.72it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▊     | 356/430 [02:11<00:27,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 357/430 [02:11<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|████████████████████████▉     | 358/430 [02:12<00:26,  2.73it/s]\u001b[A\r\n",
      "Iteration:  83%|█████████████████████████     | 359/430 [02:12<00:26,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████     | 360/430 [02:12<00:25,  2.73it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▏    | 361/430 [02:13<00:25,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 362/430 [02:13<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  84%|█████████████████████████▎    | 363/430 [02:13<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 364/430 [02:14<00:24,  2.72it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▍    | 365/430 [02:14<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 366/430 [02:15<00:23,  2.71it/s]\u001b[A\r\n",
      "Iteration:  85%|█████████████████████████▌    | 367/430 [02:15<00:23,  2.72it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 368/430 [02:15<00:22,  2.70it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▋    | 369/430 [02:16<00:23,  2.60it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▊    | 370/430 [02:16<00:23,  2.61it/s]\u001b[A\r\n",
      "Iteration:  86%|█████████████████████████▉    | 371/430 [02:16<00:22,  2.64it/s]\u001b[A\r\n",
      "Iteration:  87%|█████████████████████████▉    | 372/430 [02:17<00:21,  2.67it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 373/430 [02:17<00:21,  2.69it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████    | 374/430 [02:18<00:20,  2.70it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 375/430 [02:18<00:20,  2.71it/s]\u001b[A\r\n",
      "Iteration:  87%|██████████████████████████▏   | 376/430 [02:18<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 377/430 [02:19<00:19,  2.72it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▎   | 378/430 [02:19<00:19,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▍   | 379/430 [02:19<00:18,  2.71it/s]\u001b[A\r\n",
      "Iteration:  88%|██████████████████████████▌   | 380/430 [02:20<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▌   | 381/430 [02:20<00:18,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 382/430 [02:20<00:17,  2.72it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▋   | 383/430 [02:21<00:17,  2.73it/s]\u001b[A\r\n",
      "Iteration:  89%|██████████████████████████▊   | 384/430 [02:21<00:16,  2.74it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▊   | 385/430 [02:22<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|██████████████████████████▉   | 386/430 [02:22<00:16,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 387/430 [02:22<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████   | 388/430 [02:23<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  90%|███████████████████████████▏  | 389/430 [02:23<00:15,  2.72it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▏  | 390/430 [02:23<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 391/430 [02:24<00:14,  2.71it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▎  | 392/430 [02:24<00:14,  2.69it/s]\u001b[A\r\n",
      "Iteration:  91%|███████████████████████████▍  | 393/430 [02:25<00:13,  2.70it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▍  | 394/430 [02:25<00:13,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▌  | 395/430 [02:25<00:12,  2.72it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 396/430 [02:26<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  92%|███████████████████████████▋  | 397/430 [02:26<00:12,  2.71it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 398/430 [02:26<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▊  | 399/430 [02:27<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 400/430 [02:27<00:11,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|███████████████████████████▉  | 401/430 [02:27<00:10,  2.72it/s]\u001b[A\r\n",
      "Iteration:  93%|████████████████████████████  | 402/430 [02:28<00:10,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████  | 403/430 [02:28<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▏ | 404/430 [02:29<00:09,  2.73it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 405/430 [02:29<00:09,  2.74it/s]\u001b[A\r\n",
      "Iteration:  94%|████████████████████████████▎ | 406/430 [02:29<00:08,  2.73it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 407/430 [02:30<00:08,  2.74it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▍ | 408/430 [02:30<00:08,  2.72it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 409/430 [02:30<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  95%|████████████████████████████▌ | 410/430 [02:31<00:07,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 411/430 [02:31<00:06,  2.72it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▋ | 412/430 [02:32<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▊ | 413/430 [02:32<00:06,  2.73it/s]\u001b[A\r\n",
      "Iteration:  96%|████████████████████████████▉ | 414/430 [02:32<00:05,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|████████████████████████████▉ | 415/430 [02:33<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 416/430 [02:33<00:05,  2.72it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████ | 417/430 [02:33<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 418/430 [02:34<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  97%|█████████████████████████████▏| 419/430 [02:34<00:04,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 420/430 [02:34<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▎| 421/430 [02:35<00:03,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▍| 422/430 [02:35<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  98%|█████████████████████████████▌| 423/430 [02:36<00:02,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▌| 424/430 [02:36<00:02,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 425/430 [02:36<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▋| 426/430 [02:37<00:01,  2.72it/s]\u001b[A\r\n",
      "Iteration:  99%|█████████████████████████████▊| 427/430 [02:37<00:01,  2.73it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▊| 428/430 [02:37<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|█████████████████████████████▉| 429/430 [02:38<00:00,  2.72it/s]\u001b[A\r\n",
      "Iteration: 100%|██████████████████████████████| 430/430 [02:38<00:00,  2.71it/s]\r\n",
      "Epoch: 100%|█████████████████████████████████████| 2/2 [05:17<00:00, 158.57s/it]\r\n",
      "100%|████████████████████████████████████| 3535/3535 [00:00<00:00, 14374.97it/s]\r\n",
      "convert squad examples to features: 100%|██| 3535/3535 [00:06<00:00, 557.86it/s]\r\n",
      "add example index and unique id: 100%|██| 3535/3535 [00:00<00:00, 572709.05it/s]\r\n",
      "Evaluating: 100%|███████████████████████████████| 56/56 [00:06<00:00,  8.12it/s]\r\n"
     ]
    }
   ],
   "source": [
    "train_file = \"original/train.json\"\n",
    "predict_file = \"original/test.json\"\n",
    "run_script(train_file, predict_file, batch_size, lr, epochs, max_seq_len, doc_stride)\n",
    "!mv results/predictions_.json results/test_predictions.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submission"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(selected):\n",
    "    return \" \".join(set(selected.lower().split()))\n",
    "\n",
    "# Copy predictions to submission file.\n",
    "predictions = json.load(open('results/test_predictions.json', 'r'))\n",
    "submission = pd.read_csv(open('/kaggle/input/tweet-sentiment-extraction/sample_submission.csv', 'r'))\n",
    "for i in range(len(submission)):\n",
    "    id_ = submission['textID'][i]\n",
    "    if post_processing and (pd_test['sentiment'][i] == 'neutral' or len(pd_test['text'][i].split()) < 4): # post-processing\n",
    "        submission.loc[i, 'selected_text'] = f(pd_test['text'][i])\n",
    "    else:\n",
    "        submission.loc[i, 'selected_text'] = f(predictions[id_])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>textID</th>\n",
       "      <th>selected_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11aa4945ff</td>\n",
       "      <td>i wish</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>fd1db57dc0</td>\n",
       "      <td>done.haha. i'm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2524332d66</td>\n",
       "      <td>concerned i'm</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0fb19285b2</td>\n",
       "      <td>worry. hey guys working need to it's no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>e6c9e5e3ab</td>\n",
       "      <td>26th february</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       textID                            selected_text\n",
       "0  11aa4945ff                                   i wish\n",
       "1  fd1db57dc0                           done.haha. i'm\n",
       "2  2524332d66                            concerned i'm\n",
       "3  0fb19285b2  worry. hey guys working need to it's no\n",
       "4  e6c9e5e3ab                            26th february"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "submission.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the submission file.\n",
    "submission.to_csv('submission.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
